GPUs have been favored for training deep learning models due to their highly parallelized architecture. As a result, most studies on training optimization focus on GPUs. There is often a trade-off, however, between cost and efficiency when deciding on how to choose the proper hardware for training. In particular, CPU servers can be beneficial if training on CPUs was more efficient, as they incur fewer hardware update costs and better utilizing existing infrastructure. This paper makes several contributions to research on training deep learning models using CPUs. First, it presents a method for optimizing the training of deep learning models on Intel CPUs and a toolkit called ProfileDNN, which we developed to improve performance profiling. Second, we describe a generic training optimization method that guides our workflow and explores several case studies where we identified performance issues and then optimized the Intel Extension for PyTorch, resulting in an overall 2x training performance increase for the RetinaNet-ResNext50 model. Third, we show how to leverage the visualization capabilities of ProfileDNN, which enabled us to pinpoint bottlenecks and create a custom focal loss kernel that was two times faster than the official reference PyTorch implementation.

In the release of Python 2.0, contributions from Intel using Intel Extension for PyTorch, oneAPI Deep Neural Network Library (oneDNN) and additional support for Intel CPUs enable developers to optimize inference and training performance for artificial intelligence (AI). As part of the PyTorch 2.0 compilation stack, the TorchInductor CPU backend optimization by Intel Extension for PyTorch and PyTorch ATen CPU achieved up to 1.7 times faster FP32 inference performance when benchmarked with TorchBench, HuggingFace and timm.1 This update brings notable performance improvements to graph compilation over the PyTorch eager mode. Notices & Disclaimers Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details.

New features for AI developers in research and production environments optimize performance. In the release of Python 2.0, contributions from Intel using Intel® Extension for PyTorch , oneAPI Deep Neural Network Library (oneDNN) and additional support for Intel® CPUs enable developers to optimize inference and training performance for artificial intelligence (AI). As part of the PyTorch 2.0 compilation stack, the TorchInductor CPU backend optimization by Intel Extension for PyTorch and PyTorch ATen CPU achieved up to 1.7 times faster FP32 inference performance when benchmarked with TorchBench, HuggingFace and timm.1 This update brings notable performance improvements to graph compilation over the PyTorch eager mode. Other optimizations include: More: Read…

End-to-end (E2E) artificial intelligence (AI) pipelines are composed of several stages including data preprocessing, data ingestion, defining and training the model, hyperparameter optimization, deployment, inference, postprocessing, followed by downstream analyses. To obtain efficient E2E workflow, it is required to optimize almost all the stages of pipeline. Intel Xeon processors come with large memory capacities, bundled with AI acceleration (e.g., Intel Deep Learning Boost), well suited to run multiple instances of training and inference pipelines in parallel and has low total cost of ownership (TCO). To showcase the performance on Xeon processors, we applied comprehensive optimization strategies coupled with software and hardware acceleration on variety of E2E pipelines in the areas of Computer Vision, NLP, Recommendation systems, etc. We were able to achieve a performance improvement, ranging from 1.8x to 81.7x across different E2E pipelines. In this paper, we will be highlighting the optimization strategies adopted by us to achieve this performance on Intel Xeon processors with a set of eight different E2E pipelines.

We Need Your Support: This site is primarily supported by advertisements. Ads are what have allowed this site to be maintained on a daily basis for the past 18 years. We do our best to ensure only clean, relevant ads are shown, when any nasty ads are detected, we work to remove them ASAP. If you would like to view the site without ads while still supporting our work, please consider our ad-free Phoronix Premium.

In the last post, I introduced Intel Distribution of Modin and Intel Extension for Scikit-learn, integral parts of the Intel oneAPI AI Analytics Toolkit, and the overall Intel AI Software suite. Let's take a closer look at Modin and Scikit-learn extensions through this tutorial. The objective of this guide is to highlight how Modin and Scikit-learn extensions are a drop-in replacement for stock Pandas and Scikit-learn libraries. You can try this tutorial either in Intel DevCloud or your workstation. For this tutorial, I provisioned an e2-standard-4 VM on Google Compute Engine with 4 vCPUs and 16GB RAM based on the Intel Broadwell platform.

"Fantastic! How fast can we scale?" Perhaps you've been fortunate enough to hear or ask that question about a new AI project in your organization. Or maybe an initial AI initiative has already reached production, but others are needed -- quickly. At this key early stage of AI growth, entesrprises and the industry face a bigger, related question: How do we scale our organizational ability to develop and deploy AI? Business and technology leaders must ask: What's needed to advance AI (and by extension, data science) beyond the "craft" stage, to large-scale production that is fast, reliable, and economical? The answers are crucial to realizing ROI, delivering on the vision of "AI everywhere", and helping the technology mature and propagate over the next five years.